A Multimodal Approach to Audiovisual Text-to-Speech Synthesis

Oral speech has always been the most important means of communication between humans. When a message is conveyed using oral speech, it is encoded in two separate signals: an auditory speech signal and a visual speech signal. The auditory speech signal consists of a series of speech sounds that are produced by the human speech production system. In order to generate different sounds, the parameters of this speech production system are varied. Since some of the human articulators are visible to an observer (e.g., the lips, the teeth and the tongue), while uttering the speech sounds the variations of these visible articulators define the visual speech signal. It is well known that an optimal conveyance of the message is possible only when both the auditory and the visual speech signals are perceived by the receiver.